Open Source


There’s been a lot of talk about the new SIP stack that is being worked on for Asterisk. The SIP stack is tightly connected to the RTP media subsystem and that’s a part of Asterisk I’ve spent a lot of time with lately. Here’s a few items I’ve been working on and hope to get into Asterisk 12:

Pinefool – Packet loss concealment and more

This branch improves audio handling. If configured  it adds packet loss concealment which improves recordings quite a lot
when there’s packet loss in the network. This is a very simple form of packet loss concealment, which I call the “Poor Man’s PLC”. If there is packet loss, Asterisk will just copy the previously received audio frame to conceal it. Asterisk already has a more advanced Packet Loss Concealment function, that’s hard to activate when you just forward packets in the RTP layer without any transcoding.

This patch also fixes some issues with packet reordering and loss. Previously Asterisk was masqing it, so two endpoints talking G.729 couldn’t fix the packet loss in their code. With Pinefool, Asterisk will keep the properties of the inbound stream on the outbound stream. A lost packet in the incoming stream will cause a lost packet in the outbound stream. Packets arriving out of order will still be out of order on the outbound stream, making it possible for the receiving end point, which in most cases has a jitter buffer with packet loss concealment, to sort out the mess and produce high-quality audio. I have to acknowledge a lot of help in brainstorms over a lot of chat sessions from Martin Festr Vit, the developer of VoipMonitor.org.

Benefit: Improved audio in lossy networks, better quality recordings

Pinefrog – Improved RTCP and call quality records (CQR)

The Pinefrog branch adds reporting of RTCP for streams that have it. By saving data about each call leg in database or outputting it on the manager port, a third-party application can monitor the health of SIP trunks and possibly calculate MOS score on calls. With an intelligent application, the data could also be used to disable trunks that doesn’t work as expected or select codecs based on network quality.

The CQR, Call Quality Records, are stored in any database supported by the ARA, Asterisk Realtime Architecture. This code has been in production in a large set of different sites, from carriers to call centers, for many years in the 1.4 branch.

Benefit: Being able to manage QoS and make automatic decisions based on this input in 3rd party apps

Pinequeue – background audio processing for queues

This patch implements a new audio backend for queues. Previously agents had to wait for a prompt to be played
for a caller in the queue before the phone gave a ring. With Pinequeue audio is handled in the background
and we will interrupt any prompts played as soon as an agent answers the call.

Benefit: No agents waiting to answer customers that in fact is waiting for a free agent.

Rana – Adjust the DTMF Duration and keep it under control

Fixes a lot of problems with DTMF passed through Asterisk. The duration wasn’t handled properly, which resulted in large differences between the incoming DTMF tone and the tone sent on the outbound call leg.

Benefit: 200 ms DTMF will not become 500 ms or even worse.

Roibus – Implements support of comfort noise (inbound only)

With this patch, Asterisk will negotiate and support comfort noise support on incoming media. Comfort noise goes hand in hand with silence suppression in a device. If a device recognizes silence, it stops sending audio frames and instead instructs the other end to generate replacement audio called “comfort noise”.

This patch is not yet complete. Asterisk will not send cng on the outbound stream yet. Regardless, the patch is already saving a lot of bandwidth and is a requirement for Microsoft Lync trunks too. Primarily it is a bandwidth saving issue. Previously Asterisk did not support it so for a 64 K codec we always send 64k plus over head. Coming work will add support for silence suppression in Asterisk, so that Asterisk can make a decision on whether to send actual audio or suppress it. This will add overhead in your Asterisk server, as a DSP will have to be applied to the audio stream to listen in for silence. In many cases this is worthwhile, as the bandwidth saving generates so many benefits.

Benefit: Some VoIP bandwidth calculation web sites estimates that full (two-way) silence suppression and CNG saves between 30-40% of the bandwidth for a call.

Current work: Implementing full DNS SRV-based failover and load balancing in Asterisk

Currently I’m working with improving the failover and load balancing when using DNS srv records. RFC 3263 describes how this should work in SIP. The Asterisk SIP stack has limited support – just grabbing the first alternative, not failing over. With the help of a set of dialplan functions, you can organize failover for the first INVITE in a call, but no failover during the call. This will have to be fixed.

The pinefrog, darjeeling, pinequeue, rana and roibus branches are in production in at least two large carriers and in some of our call centers. Pinefool is pretty new, we’re testing it in a few sites now. They are written for Asterisk 1.4 or Asterisk 1.8 with ports to many versions. Together with the Digium development team, I am working to make this code part of Asterisk 12.

The combination of pinefool, pinefrog, roibus and rana will improve your media handling vastly.

Check your bug tracker and sales records to see if you haven’t got requests for these kind of
features! If you have then download the branches from my subversion tree and try them out! 

/O

PS. Every branch has a README file for the branch in the main directory. Don’t forget to read it. Thanks!

Last weekend I attended a session about Homer Sipcapture at Fosdem in Brussels. Homer is a software that is built in to Kamailio. In Kamailio has there are two modules that is specially adopted for use with this 3rd party Open Source software – Homer SIPcapture.  Homer was awarded a Kamailio Award last year for it’s unique innovation. With Kamailio and Homer, you can capture SIP messages from a running Kamailio production server or from a mirrored port in a switch in your network. What do you get? A searchable database of your SIP traffic and easy to understand visual diagrams of individual SIP sessions. All in a web interface powered by a database of your traffic. I installed it and after a few hours it was a critical tool in my network, helping me to solve problems.

Two components – sender and receiver

Homer has two major components. The packet sender captures traffic and forwards to a receiver. You can use Kamailio in both places, but you can also enable a sender within FreeSwitch as well as use the Homer capture sender that reads off a network interface and sends to the receiver. To use the Homer software with Kamailio, you need a database server (with capacity to handle your traffic) and a web server with PHP support. (more…)

The very first SIP-compatible software I installed myself was SER, Sip Express Router, from iptel.org, a spinoff from the Fokus research institute in Berlin. During the years I have worked with SER and OpenSER/Kamailio, I’ve become friends with many of the developers and have contributed with documentation and training. The team gathers world leading SIP experts and developers, so it’s a joy to meet them and brainstorm and share experiences. In almost all my installations and platforms, a SIP proxy is at the core of the real time multimedia platform. My choice has always been SER/OpenSER/Kamailio.

Starting as a SIP proxy, moving on as an extensible SIP server

Sip Express Router started as an Open Source SIP Proxy. Today we see Kamailio, based on the common sip-router.org core, as an extensible SIP server. With an embedded presence server, http support and interfaces to python and lua for scripting, it’s a platform that can handle many different roles in a large-scale SIP architecture. IPv4 and IPv6 support from the beginning, security integrated and a very active development team. Everything you need to build a standard-compliant cool network for your organization’s real-time communication. 10 years of success.

Jubilee September 2nd, 2011 where it all started

The 3rd of September marks 10 years since the start. Reason to celebrate, don’t you think?

 

We will celebrate on Friday, September 2nd and into the night. FhG Fokus Research Institute hosts a one-day conference session with all the core people. If you have any interest in SIP and these platforms, make sure you will be there.

The event is organized with sponsorships from FhG Fokus Institute, Asipto, Amooma and TeamForrest. At the same time local events will be organized in Barcelona, Spain and Vienna, Austria.  Read the agenda and register today!

A friend of mine, Jacob Schlyter, has not only been busy working on DNSsec around the globe. He has also been co-writing a new draft in the IETF DANE working group. He blogs on kirei.se:

The working group’s first draft describes how to use DNSSEC to associate certificates with domain names For TLS. In this context, a certificate association is a secure binding between a DNS name, TLS/DTLS transport protocol (i.e., TCP, UDP or SCTP) and port number, to an end entity’s certificate or to a certification authority’s certificate.

This proposal builds a distributed PKI based on DNS.  If you want to call me in the voip-forum.com domain, you will be able to get my certificate chain from DNS and set up a secure and trusted session. The current PKI model used in the web delivers encryption – but few bothers with checking identity of the other end, check if the certificate chain is trusted or even if a certificate is revoked.

In SIP, we can do better and really kick off DANE. I will work on this and write more as I make progress in trying to understand the details.

For many years I have believed that DNSsec is a platform that will change the trust model on the net. Now, the light is on the ICANN, IANA and the TLD registries. Will they earn our trust?  Will they be better than the current set of web CAs that someone else approved for all of us? If not, we will have to start building a distributed trustworthy root soon. The technology is getting there, so the focus needs to move to the people and organizations that will be the new platform. These are interesting times. Let’s start discussing SIPdane and get a few implementations running soon!

FOSS Stockholm is a group of Open Source developers, users, advocates and fans that meet and talk about various Open Source related issues. Today we had a lot of good talks on Open Source in development countries and it’s affect on democracy, about Open Source project management and other interesting topics. I had a quick overview of realtime communication security in order to stimulate a discussion. The slides are now published on Slideshare.

Realtime communication security – SIP, XMPP and others

View more presentations from Olle E Johansson.

You are drafted for my IPv6 SIP task force! Here’s our action points:

  • Review IETF documents on SIP IPv6  migration. Are all depending on all devices having dual stack? If so, does it really work?
  • Test IPv6-only solutions. Do they work? Is it possible to use Asterisk or Kamailio for reaching IPv4 SIP devices?
  • Test DNS SRV records. Are they usable to indicate your preference for IPv6 or IPv4 calls to your domain? Will devices support it?
  • Check SIP service providers – do they support IPv6 today?
  • Check Open Source solutions – is the IPv6 support complete? Does it work both in dual stack and single IPv6 stack mode?
  • Check phone manufacturers – does their products support IPv6?

Are there other questions to sort out? Is so, let me know. And let’s make 2011 the year when the SIP world embraced IPv6!

Jon Postel captured an approach to make the Internet a more robust place for applications to live in – and added it to RFC793 (the definition of TCP at the time) :

“TCP implementations will follow a general principle of robustness: be conservative in what you do, be liberal in what you accept from others.”.

Robert Sparks has written an article on the SIP Sessions blog about the importance of this for SIP implementations. And he should know. Robert has been managing 25 interoperability sessions with SIP developers. Endless patience with developers that claim that their interpretation of the RFC is correct and that the other side is damn wrong and that’s exactly the reason why their implementation fails the call, the registration or the chat session. I’ve heard Robert saying to me and others many, many times something like: “Yes, there is an issue here, but your app should be more generous and try to handle the situation.” Robert summarizes his thoughts this way:

“Real systems will have imperfections. Systems that use the robustness principle make it less likely that those imperfections will result in failure. Introducing elements that don’t follow the principle negates that robustness, and in real systems,  will cause those systems to fail.”

Read this article named “The Robustness Principle Has Two Parts” 

I am working with two very different implementation projects this winter. One is about using Asterisk in large call center environments. Another has a goal of building a SIP network for 15.000 phones in a university.

Asterisk for large call centers – full control

The call center platform is focused on a lot of PBX functionality. Every agent stays connected to a conference session, where customers are connected and disconnected, recording is enabled and disabled, dtmf is used to control various services and both AGI and AMI (the manager interface) are both being used heavily to integrate with a controlling application. Asterisk is in full control of each and every call, all the time, but the application controls the flow of the calls. The dialplan is very small for a large-scale Asterisk installation.

Large scale SIP networks – Asterisk on the edge, providing services

For the university, scaling is important. It’s a plain SIP network, with two computer centers in different buildings. All accounts are managed by LDAP, there’s no account defined locally in the VoIP platform. SIP proxys (Kamailio) rule this network and DNS is used for failover. Many services like call transfer, three-party conference calls and call forwarding will be handled by the phones. Asterisk serves as gateways to the old world, Nortel systems, and feature servers for IVR, voicemail and switchboard. No single server is in control of any call.

The power of open standards and open source

Two completely different designs, both made possible by Open Source and Open Standards. Both of them needs to scale. Both of them needs failover, redundancy and stability. And in both cases, we’re replacing expensive legacy telecom equipment with new platforms that will cost less to operate, that has a higher degree of interoperability and much more functionality than the previous solutions. Open telephony wins.

I’ve created a test branch for patches hidden in several Asterisk development branches – all based on Asterisk 1.4

  • RTCP improvements from pinefrog-1.4
  • “Sip show chanstats” cli command
  • The branch pinequality-* giving you the manager “sipchannel” event to check QoS

This branch is now open for testing and I need feedback. Among the improvements you’ll find:

  • Manager QoS events during a call and after a call
  • Improved RTCP – it now works for p2p bridge in RTP, which means that we will get RTCP stats for many, many more sip calls
  • RTCP over NAT improvements – if Asterisk is behind NAT, we will now kick-start RTCP from the remote end by sending a first “emtpy” RTCP packet to open a NAT port.
  • QoS reports to realtime storage after each call – one report per call leg (The amount of data and the names will change)

The reason that I store  QoS data in realtime, is that the CDR is usually gone or frozen at the time that we freeze the RTP channels and get the last QoS data. The QoS reports can’t thus be included in CDR, you have to merge it in automatically later in your database.There’s still a lot to do, but please test it so I get some sort of feedback.For testing, don’t forget to run the “rtcp debug” cli command so you can see what’sgoing on in the RTCP channel.

FAQ

  • Yes, this work will be ported to trunk and hopefully merged soon.
  • No, we don’t support RTCP XR or MOS in this work
  • No, I have no reason or funding to adapt it to 1.6.x at this point.
  • No, the RTPAUDIOQOS channel variable is not changed. You will get more data than before – for many more calls.

This work is funded to 20% by companies in the community. If you want to cover the80% that’s still not funded, please contact me by e-mail: oej@edvina.net.
URL:  
http://svn.asterisk.org/svn/asterisk/team/oej/pinefrog-deluxe-rtcp-test/

Kamailio - the successor to OpenSER – has been released in a new version, 3.0. This version is based on the merge between SER, SIP Express Router, and OpenSER code bases and developer teams. It’s a SIP server that you can run as a SIP proxy, a session border controller, a SIP load balancer or SIP application server. In version 3.0 you have the power of all the modules and applications in OpenSER and the raw strength of the SER core in one product. Go check it out today!

Next Page »