Asterisk news


There’s been a lot of talk about the new SIP stack that is being worked on for Asterisk. The SIP stack is tightly connected to the RTP media subsystem and that’s a part of Asterisk I’ve spent a lot of time with lately. Here’s a few items I’ve been working on and hope to get into Asterisk 12:

Pinefool – Packet loss concealment and more

This branch improves audio handling. If configured  it adds packet loss concealment which improves recordings quite a lot
when there’s packet loss in the network. This is a very simple form of packet loss concealment, which I call the “Poor Man’s PLC”. If there is packet loss, Asterisk will just copy the previously received audio frame to conceal it. Asterisk already has a more advanced Packet Loss Concealment function, that’s hard to activate when you just forward packets in the RTP layer without any transcoding.

This patch also fixes some issues with packet reordering and loss. Previously Asterisk was masqing it, so two endpoints talking G.729 couldn’t fix the packet loss in their code. With Pinefool, Asterisk will keep the properties of the inbound stream on the outbound stream. A lost packet in the incoming stream will cause a lost packet in the outbound stream. Packets arriving out of order will still be out of order on the outbound stream, making it possible for the receiving end point, which in most cases has a jitter buffer with packet loss concealment, to sort out the mess and produce high-quality audio. I have to acknowledge a lot of help in brainstorms over a lot of chat sessions from Martin Festr Vit, the developer of VoipMonitor.org.

Benefit: Improved audio in lossy networks, better quality recordings

Pinefrog – Improved RTCP and call quality records (CQR)

The Pinefrog branch adds reporting of RTCP for streams that have it. By saving data about each call leg in database or outputting it on the manager port, a third-party application can monitor the health of SIP trunks and possibly calculate MOS score on calls. With an intelligent application, the data could also be used to disable trunks that doesn’t work as expected or select codecs based on network quality.

The CQR, Call Quality Records, are stored in any database supported by the ARA, Asterisk Realtime Architecture. This code has been in production in a large set of different sites, from carriers to call centers, for many years in the 1.4 branch.

Benefit: Being able to manage QoS and make automatic decisions based on this input in 3rd party apps

Pinequeue – background audio processing for queues

This patch implements a new audio backend for queues. Previously agents had to wait for a prompt to be played
for a caller in the queue before the phone gave a ring. With Pinequeue audio is handled in the background
and we will interrupt any prompts played as soon as an agent answers the call.

Benefit: No agents waiting to answer customers that in fact is waiting for a free agent.

Rana – Adjust the DTMF Duration and keep it under control

Fixes a lot of problems with DTMF passed through Asterisk. The duration wasn’t handled properly, which resulted in large differences between the incoming DTMF tone and the tone sent on the outbound call leg.

Benefit: 200 ms DTMF will not become 500 ms or even worse.

Roibus – Implements support of comfort noise (inbound only)

With this patch, Asterisk will negotiate and support comfort noise support on incoming media. Comfort noise goes hand in hand with silence suppression in a device. If a device recognizes silence, it stops sending audio frames and instead instructs the other end to generate replacement audio called “comfort noise”.

This patch is not yet complete. Asterisk will not send cng on the outbound stream yet. Regardless, the patch is already saving a lot of bandwidth and is a requirement for Microsoft Lync trunks too. Primarily it is a bandwidth saving issue. Previously Asterisk did not support it so for a 64 K codec we always send 64k plus over head. Coming work will add support for silence suppression in Asterisk, so that Asterisk can make a decision on whether to send actual audio or suppress it. This will add overhead in your Asterisk server, as a DSP will have to be applied to the audio stream to listen in for silence. In many cases this is worthwhile, as the bandwidth saving generates so many benefits.

Benefit: Some VoIP bandwidth calculation web sites estimates that full (two-way) silence suppression and CNG saves between 30-40% of the bandwidth for a call.

Current work: Implementing full DNS SRV-based failover and load balancing in Asterisk

Currently I’m working with improving the failover and load balancing when using DNS srv records. RFC 3263 describes how this should work in SIP. The Asterisk SIP stack has limited support – just grabbing the first alternative, not failing over. With the help of a set of dialplan functions, you can organize failover for the first INVITE in a call, but no failover during the call. This will have to be fixed.

The pinefrog, darjeeling, pinequeue, rana and roibus branches are in production in at least two large carriers and in some of our call centers. Pinefool is pretty new, we’re testing it in a few sites now. They are written for Asterisk 1.4 or Asterisk 1.8 with ports to many versions. Together with the Digium development team, I am working to make this code part of Asterisk 12.

The combination of pinefool, pinefrog, roibus and rana will improve your media handling vastly.

Check your bug tracker and sales records to see if you haven’t got requests for these kind of
features! If you have then download the branches from my subversion tree and try them out! 

/O

PS. Every branch has a README file for the branch in the main directory. Don’t forget to read it. Thanks!

A new Asterisk is being worked on. I look forward to it!

A new Asterisk is being worked on. I look forward to it!

I must say that I’m impressed with the new project leader for Asterisk, Matt Jordan. There are big changes happening under his leadership and I don’t think everyone has understood the huge tasks he’s taken on with his team. I’m not sure I’ve understood all of it either.

Here’s a list of things I’ve seen on the mailing lists:

  • Dialog with the community is much improved. Brainstorms happen in the open and things are developed in the open. This is a big change. I just hope I had more time to engage, but that’s my problem. The community seems a bit surprised too. I expected much more participation. We need to relearn how to contribute and discuss, now that we’re allowed to do that again.
    Digium had to re-learn how to interact with the community – but that goes two ways. The community needs to stand up to the challenge. Get involved.
  • Old issues that we wanted to forget are attacked and fixed. Like Masquerade and bridging – it’s all about to change. When I worked with Terry Wilson on the SIP transfer code in 1.2 we learned to hate Masquerade. That’s years ago and the masquerade is still around, annoying all Asterisk developers. It’s a complex operation that, well, causes a lot of issues. Time to get rid of it and get a modern multichannel bridge active in every call.
  • Manager is being changed dramatically.  Manager is getting an overhaul into something more logical for the current design of Asterisk and languages and methods used to build AMI applications. Yes, this will affect everyone’s 3rd party applications. Applications needs to move on, sometimes we just can’t be backwards compatible. Most applications doesn’t support punch cards and output exceed 80 characters per line quite often… Asterisk is ten years old and it’s time for some new designs. Asterisk 11 is a LTS that will live for a long time, giving developers time to adopt to the new Asterisk.
  • The SIP Channel is rewritten. This is a gigantic project and has been needed for almost ten years. I never thought we could get funding for a complete rewrite, so I opted for remaking the current code in my old Codename Pineapple project that never got fully funded at the time. I have tons of opinions about a new SIP channel, but have no resources to really participate, just add a comment here and there like a grumpy old grandpa…
    I wish the developers follow my SIP2012 efforts (soon to be renamed SIP2014) and learn modern SIP and have the new SIP functions as part of the design. I do hope that they DO NOT base it on the current SIP channel that lacks transaction support, has poor ideas on SIP branching and forking and… You know. The new SIP implementation is required to work in larger SIP networks with proxies, unlike the old SIP channel. We need domain support, we need SIP URI support, we need security that works. That means that the developers need a better understanding of SIP than has been the tradition in the Asterisk development team. They’re always free to contact me with questions, which would be a positive change too.
  • New APIs are developed. Manager and AGI and ExternIVR are things that have happened and evolved but was never part of a consolidated effort to create a good unified API to Asterisk. We’ve discussed this at many Astridevcons under different names – PineMango was one of them. For Asterisk to survive, we need a modern API.
  • Realtime is changing. Josh Colp is working on a new object handling system, again something discussed many times that finally happens. We do need a replacement of the poor realtime architecture that has been extended beyond control. We need to be able to use an API to manage in-memory objects in real time. It seems to me that this is exactly what File is working on.

Doing all of this at the same time sounds to me like someone wanting to make a stand. If all of these project will be completed we will have a brand new Asterisk coming out. Kevin changed almost all the internals for the better . Matt and his team will change a lot on this list (and propably much more), things that change the functionality for the better, not just improve stability. Both are needed, but from a marketing standpoint all these changes are big things that can make a real splash at product release.
In the future I hope we can have time to spend on innovation, new things that will amaze all of us, that fits into this new Asterisk. Not just rewrites and stabilization.

The Matt Jordan challenge – what’s the community response?

Inspiration for this new stuff needs to come from the community. We expect too much work being done by Digium – even though they keep showing us that they still invest heavily in Asterisk, even in bad times. Can the community step up and assist in the redesign of our good old Asterisk? There are so many companies out there getting their living out of Asterisk. Fund a developer, contribute with resources, test, participate. The Matt Jordan challenge is on!

Thank you Digium

A big thank you to Digium and Matt and your team for all this work. It sounds like you are having fun. We did need more fun in Asterisk. We did need more open communication with the community, improved interaction – and I’m very happy to see the changes happen. The next step is a vision for Asterisk – that’ll be the OEJ challenge. Where is Asterisk going?

 

Realtime communication in web browsersVideo sessions in the browser opens up for a lot of new applications. While this has been working with various plugins, new opportunities will open up when it becomes part of your standard web browser. HTML5 introduced native video in the browser. New standardization are looking into peer2peer multimedia sessions in the browser, not just broadcast. This opens up for a large set of applications, from “click2call” without plugins to video chats like in Facebook and Google+. This is exciting. And all the old SIP architects seems to have jumped onto this project, trying to get things right from start in this second generation effort. (more…)

Imagine working at your desk, getting a phone call from your friend Randy. You answer on your IP phone. Being Randy, he suddenly wants to play a new jingle he created while being in the mood the day before. The phone speaker is not a good device for a cool guitar riff – is it? On the same desk you have your PC with a softphone that supports HD voice and really cool loudspeakers. Why not transfer the audio to the PC while still using the phone’s microphone? After that, you want to play a video for Randy. Now you want to add media from your laptop to the call – while the call is still managed by the IP phone.

This is a true multimedia session, that is not that far away. But it’s too complex to manage in the SIP protocol today. I just found out about the SPLICES working group in the IETF that is working on this issue. I also know that the Asterisk development team is working on a new media architecture - something we’ve needed for many years. My presentation about the new media architecture from Astridevcon 3 years ago talks about being able to add streams, have multiple streams of the same kind and remove streams dynamically during a call. SPLICES, when finished, will help us implement it in a really cool way in SIP.

Another mailing list to follow.

Do you know another thing? Randy and I wouldn’t be able to have this call as easily if we did not use SIP over IPv6. Doing a call like the one above over two NATs would be awfully complex…

 

Sollentuna, Geneva and Prague, April 1st, 2011: The IETF and the IPv6 Forum today launched SIP-six, the new version of SIP that cleans up a lot of misunderstandings and greatly simplifies implementations of SIP. 

  • ”We realize that 99% of the SIP users use SIP for PSTN phone calls. The original SIP standards was written with other applications in mind, a vision that never came true.” said Bob Plug, area director in the IETF. ”That’s why we sat down and said to our selves that this should be way more simple.”

The SIP-six standard totally removes the dependency of domains and URI syntax. There’s no longer any point in using this, since everyone seems to think that IP addressing is more than enough. The new standard use part of the vast IPv6 address space to incorporate the E.164 phone number as end point addresses. This is the reverse of the reverse phone number usage in the enum standard, which is no longer needed in SIP-six.

By using IPv6 mobile IP, phone users register their phones and get access to their phone number. Users that need security can easily integrate IPsec into their setup. Media and signalling uses the same addressing scheme and is mixed so that both SIP-six, RTP and RTCP only uses one port address – but in this case a single port address with a 32 bit subaddress identifying the media stream. This finally solves a lot of the firewall traversal issues that SIP v2.0 had. With the combination of mobile IP and use of public IP addresses NAT traversal won’t be an issue.

The testbed for SIP-six has been running for a year at six choosen large SIP carriers, with the assistance of Edvina AB in Sweden and ViaGenius in Montreal, Canada. In an International effort, the testbed is today put in production and Roboid phones all over the world is automatically connected to this worldwide network.

 

  • -”We have for a long time pointed out that IPv6 is not only a replacement of IPv4, it’s an enhancement that is the breeding ground for new types of applications.” says Olle E. Johansson, chairman of the SIP-six working group. ”With SIP-six we really use the potential of the IPv6 address space and new technologies, like Mobile IPv6.”

  • -”While the SIP v2 protocol was developed with IPv6 in mind, it never really used the potential of the new IP protocol. With SIP-six we finally solve that issue and move SIP to new areas. By simplifying the protocol and removing DNS and URIs from the design, new developers and vendors will embrace SIP and we’ll see new applications in a short while.” says Bob Plug. ”This will be the final death blow to ISDN, since it’s an easy upgrade to add the IPv6 address prefix and continue to use the same addressing.”

SIP-six is implemented as a channel driver in Asterisk 2.0, as a replacement for SIP2.0 in Kamailio 4.0 and a channel module in FreeSwitch – all releases to be released later today. Softphones for testing will shortly be available from Blink and Zoiper.

The Session Initiation Protocol was developed in the 21st century. RFC 3261 is dated June 2002. In this time, network and Internet security was nothing new. Still, this is the year 2011 and most phones and servers still lack important security features. It’s not a lack of security technology and solution proposals that is the problem. It’s a lack of requirements from the market. And two paradigms that yet again meet – the netheads and the bellheads. In this article, I will try to give you an overview of the issues at hand without going to far into all the technichal details. (more…)

Building services in Asterisk is about combining a lot of small services into larger building blocks. The Asterisk platform provides you with a large set of functions, like calling out, answering, recording, playing prompts and listening for DTMF. In combination with simple logic control statements – much like any other programming language, you can use these to build complex services. Asterisk very much follow the design principles behind the UNIX operating system – small building blocks combined with a script language may build very complex larger services.

The dial plan language gets more powerful for every release

During the years, the possibilities in the dialplan have been expanded. The language have gotten a lot of new constructs and both functions and applications.  I can admit that the language is not the best of languages, but it performs the job. Compared with Asterisk 1.0 the need for using an external language in AGI is much lower with Asterisk 1.8. Thew is one big roadblock that breaks the design – voicemail.

Asterisk Voicemail is a large monolithic roadblock

The Asterisk voicemail application does not follow this design. It has built-in menus you can not change. It has built-in assumptions on how you want to handle your voicemail that you can not change. It has built-in accounts that is not very adaptable or easy to fit in with your LDAP architecture.Many people have private versions of voicemail – either hardcoded changes to the app_voicemail.c source code or something they wrote in the dial plan. A few years ago I started my own voicemail system for a customer who just needed a small voicemail-to-email solution – mini voicemail. It was a bad name, it should have been named voicemailblocks.

VM-blocks – the tools you need to build voicemail solutions

The idea was to build a series of voicemail building blocks that anyone could use to build larger customized voicemail solutions. I still think it is a good idea. After I developed the original code, channel data stores was added to Asterisk, which should make it much more easy to build the rest of these blocks. A vm-block channel data store can keep all the data about a voicemail account in memory that can be shared by the different dial plan applications.I hope that someone in the Asterisk community agrees with these ideas and continue coding on this. The first blocks are part of Asterisk 1.8. Let’s hope we get a more complete system for 1.10. If you are interested, I am open for discussion so we can agree on the blocks we need.

Asterisk was originally built as a stand-alone system, a single central point for all telephony communication. In short, a PBX. Nowadays, the Asterisk Open Source telephony server, is used in many ways – many of them not really being PBXs. All kinds of applications are being powered by Asterisk.While building applications with Asterisk, you soon realize that you’re limited to that single server. It’s hard to scale and one limiting factor is that the call state is being held in one server. Many services depend on call states – if an agent in a call center is busy, you need to find an available agent. If a trunk to the PSTN is in use, you might want to find another way out. Call states are important.Of course, the Asterisk project is now working on the long term solution, the Asterisk SCF and the applications that will be built using this framework. But that will take some time. Meanwhile, the Asterisk PBX team has been working on a few ways to distribute the call states between a group of servers. This article will describe a few of the different architectures being worked on. (more…)

At SIPit26 we began setting up a series of automated self-tests for IPv6, like we’ve done previously with SIP/TLS. We also integrated IPv6 in as many multiparty tests as possible, to see how IPv4 and IPv6 lived together.Some notes and experiences:

  • IPv4-only applications will receive IPv6 in messaging. Even if an application DO NOT support IPv6-native connections, the application will surely get IPv6 addresses in various places in the message. In SIP, a call may traverse an IPv6 proxy before reaching your IPv4 proxy or phone. Via headers will have IPv6 and maybe a record-route header too. All user agents needs to support this. We had at least one crash in a proxy that failed to parse an IPv6 address.
  • Placing an IPv4 call to a proxy that forwards the message to an IPv6 phone without handling RTP traversal leads to issues as well. The phone gets an IPv6 address in the Contact: header and failes to send the ACK properly. This happened with Asterisk. Because of parsing failure, the parser gave up and sent ACKs and BYEs to the wrong address.
  • We did successfully set up calls between IPv6 user agents using IPv6 proxys. The failures happened in the mixed scenarious.
  • When placing a call to a domain that was configured with both A and AAAA records for the SRV records, but only one of them responding, we noticed long timeouts before failover, if that even happened. Many discussions about this followed, which lead to the conclusion that this was a poorly configured domain. Some implementations have hard-coded a preference for IPv4 since IPv6 is mostly used over tunnels and add latency today. This should be user-configurable. An owner of a domain can use SRV record weights to indicate a preference to one or the other protocols, which is a better solution. If you use IPv6 over tunnels, make sure that you separate host records for A and AAAA and have a preference towards the A record hosts in your SRV records.

We do need to continue testing all kinds of migration scenarious to  be able to come up with a best current practise document. SIPit26 gave us many good experiences to build from. I hope that testing continues at SIPit27 with the new SIPit IPv6-o-matic(R)(C)(TM) and the prompts from Allison Smith!

SIPit is the main interoperability event for all things SIP. It’s organized by the SIP Forum and creates good feedback to the IETF. Asterisk has been participating in SIPit during many years and in many variants  - videocaps, Marc Blanchet’s IPv6 branch and the standard Digium releases. All these tests has lead to a large amount of improvements for Asterisk and have helped us to build a network with other developers in the business, a network which helps when we have bugs that involve interoperability with these devices or servers. SIPit has proven important for the success of Asterisk, and thus it is also important for  everyone in the Asterisk community. 

Now, when we are working on the next  long-term release (1.8) we really need to test again and make sure that we interoperate properly. New stuff, like Terry’s SRTP branch, my RTCP work and the call completion and caller ID update work needs serious testing. We need feedback to be able to fix the issues with the TCP and TLS support. (more…)

Next Page »