Asterisk news


I am working with two very different implementation projects this winter. One is about using Asterisk in large call center environments. Another has a goal of building a SIP network for 15.000 phones in a university.

Asterisk for large call centers – full control

The call center platform is focused on a lot of PBX functionality. Every agent stays connected to a conference session, where customers are connected and disconnected, recording is enabled and disabled, dtmf is used to control various services and both AGI and AMI (the manager interface) are both being used heavily to integrate with a controlling application. Asterisk is in full control of each and every call, all the time, but the application controls the flow of the calls. The dialplan is very small for a large-scale Asterisk installation.

Large scale SIP networks – Asterisk on the edge, providing services

For the university, scaling is important. It’s a plain SIP network, with two computer centers in different buildings. All accounts are managed by LDAP, there’s no account defined locally in the VoIP platform. SIP proxys (Kamailio) rule this network and DNS is used for failover. Many services like call transfer, three-party conference calls and call forwarding will be handled by the phones. Asterisk serves as gateways to the old world, Nortel systems, and feature servers for IVR, voicemail and switchboard. No single server is in control of any call.

The power of open standards and open source

Two completely different designs, both made possible by Open Source and Open Standards. Both of them needs to scale. Both of them needs failover, redundancy and stability. And in both cases, we’re replacing expensive legacy telecom equipment with new platforms that will cost less to operate, that has a higher degree of interoperability and much more functionality than the previous solutions. Open telephony wins.

Hans Petter Selasky alerted the Asterisk developer community about a potential harmful pattern in Asterisk dialplans on February 9th.  His example is as follows:

[from_sip]
exten => _X.,1,Dial(SIP/${EXTEN}@testsip)

He writes: “And if ${EXTEN} = “000@testsip&SIP/333” what turns out to happen then is similar to SQL injection :-(  ”He is exactly right. Many VoIP protocols, including IAX2 and SIP, have a very large allowed character set in the dialed extension, a character set that allows characters that are used as separators to the dial() and the queue() applications, as well as within the dialstring that these applications send to the channel drivers in Asterisk. A user can change the dial options and dial something we should not be able to dial in your system. This article describes the issue in more detail and gives you some help on how to avoid this causing trouble in your Asterisk server. (more…)

On my wishlist for 1.8 long-term-support release the #1 item is a new media negotiation framework for Asterisk.

The history – something to learn from

If we had integrated John Martin’s videocaps in time for release of 1.4 we now would have enjoyed four releases of an Asterisk with much better video support, instead of the broken support we have today. Integration of that code was denied because of plans of something greater and bigger, which we have discussed for years at many Astridevcons. Because of us denying this and other code,  outside-of-tree code has been around that fixes a lot of issues and the original Asterisk still lacks proper video support.There’s also a few codec negotation patches that has been around and maintained for years, one of the major ones being Maxim Sobolev’s patch.

What are the issues we need to solve?

There are a few main issues here:

  • Codecs: Codecs should not be handled as a bit, being turned on and off. Both audio and video codecs have attributes that we need to take care of properly
  • Call setup: When answering, the properties of the answer needs to be relayed to the calling channel, not just a control frame that says “somebody answered”.
  • Media streams: We should be prepared for multiple media streams, including multiple media streams of the same format

The new solution should be extensible, it should be easy to add both new codecs and properties.The solution has to be configurable. The way you want Asterisk to set up a bridge between two channels varies much. Some people prefer Asterisk doing transcoding some people want Asterisk to stay out of as much trouble as possible and just set up whatever is most simple. Other users just want to standardize all call legs to one type of phone, but have a different policy for other connections. There’s no one-solution-fits-all.

Asterisk needs a new media negotiation framework for continued success

We did write up a few documents on this a year ago at Astridevcon in Phoenix. I would really like to see some work on this. My personal feeling is that this is very important for the continued success of Asterisk in the marketplace. The amount of long-lived patches in this area that has been maintained for years shows that we have to do something (and that we sadly have ignored customer demand). The arrival of new codecs and new solutions, like video conferences with multiple audio and video channels, with text channels and possibly other types of channels (lika a binary channel for MSRP and digital ISDN) – all this tells us that we have to get there. (more…)

Let’s change everything – and cause no damage for the end users!

The current version of the Internet is up for a big overhaul. We have to change the whole infrastructure it runs on, the famous IP protocol. A lot of work needs to be done and it affects everyone that works with the infrastructure. The result of all the hard work? Everything will work exactly as before. Nothing gained for the end-user experience. That’s why very few are paying attention to IPv6. It is very hard to convince the persons in charge of IT projects what the benefit is compared with upgrading all PCs to Windows 7 or installing that new spam filter for e-mail.

Internet telephony needs IPv6 peer-to-peer addressing

It is easy to explain the need for a unified address space using telephony as an example. The fact that all companies and homes use a private address space that can’t be reached from the outside doesn’t matter when it comes to the old-fashioned Internet applications. The web browser contacts a server on the Internet. The e-mail client contacts an e-mail server on the Internet. The IM/Presence application contacts a server on the Internet. Nothing needs to reach in. Until you start using peer-2-peer applications. And telephony is a very common p2p application.

  • -”You know that you have a broadband router that use one IP address from the Internet, assigned to you by the provider?
  • “- “Yes”
  • - “Do you also know that the broadband router let’s all your devices on the inside share this address by setting up a private address range that can’t be reached from the outside?”
  • - “Yes”
  • - “If you add an IP phone on the inside – do you want to be able to receive calls?”
  • - “Yes”
  • - “How do you think I can call your phone  directly, if we don’t share the address space?”
  • - “I don’t know.”

With IPv6, true p2p Internet telephony will become possible. When we get rid of the need for NAT, network address translation, we can finally separate access and policy. With a unified address plan, every device on the net has the possibility of reaching every other device. Policys might prevent access and we implement these in a firewall software in the systems or in dedicated systems.Currently, in order for a phone to work on the inside of a NAT, most implementations connect to a server on the Internet. In order for an incoming call to get through, the phone or the server keeps sending empty messages. As long as these messages are sent – occupying unneeded bandwidth and resources in the network – the NAT believes there’s a communication session going on and let the messages in. The NAT itself has no policy, it just checks if there’s a client-initiated session going on or not. As long as the NAT believes there’s a session, it will forward packets from the outside to the inside device. When you have an incoming call, the server can forward an alert to the phone and the phone will start ringing. The same setup is used if your organization has a PBX system on the inside and use a SIP trunk provider on the Internet.This solution is used by a range of applications and are not unique in any way for IP telephony.  IPv6 will make it easier to enable true Internet telephony and other p2p applications, as long as your firewalls let it happen. (more…)

I’ve created a test branch for patches hidden in several Asterisk development branches – all based on Asterisk 1.4

  • RTCP improvements from pinefrog-1.4
  • “Sip show chanstats” cli command
  • The branch pinequality-* giving you the manager “sipchannel” event to check QoS

This branch is now open for testing and I need feedback. Among the improvements you’ll find:

  • Manager QoS events during a call and after a call
  • Improved RTCP – it now works for p2p bridge in RTP, which means that we will get RTCP stats for many, many more sip calls
  • RTCP over NAT improvements – if Asterisk is behind NAT, we will now kick-start RTCP from the remote end by sending a first “emtpy” RTCP packet to open a NAT port.
  • QoS reports to realtime storage after each call – one report per call leg (The amount of data and the names will change)

The reason that I store  QoS data in realtime, is that the CDR is usually gone or frozen at the time that we freeze the RTP channels and get the last QoS data. The QoS reports can’t thus be included in CDR, you have to merge it in automatically later in your database.There’s still a lot to do, but please test it so I get some sort of feedback.For testing, don’t forget to run the “rtcp debug” cli command so you can see what’sgoing on in the RTCP channel.

FAQ

  • Yes, this work will be ported to trunk and hopefully merged soon.
  • No, we don’t support RTCP XR or MOS in this work
  • No, I have no reason or funding to adapt it to 1.6.x at this point.
  • No, the RTPAUDIOQOS channel variable is not changed. You will get more data than before – for many more calls.

This work is funded to 20% by companies in the community. If you want to cover the80% that’s still not funded, please contact me by e-mail: oej@edvina.net.
URL:  
http://svn.asterisk.org/svn/asterisk/team/oej/pinefrog-deluxe-rtcp-test/

During the last week, I’ve been diving down into the RTCP protocol and Asterisk’s implementation of it. What is RTCP? In short, it’s the way to understand what’s going on with your SIP calls on the network.

VoIP relies on the IP network for media transmission

Voice over IP protcols, like SIP, needs to send the media in digital form across the network. The core IP protocols allow for packets to get lost, actually there’s nothing stopping a busy router from just deleting packets it doesn’t have time to deal with. On top of the IP protocol there are two main transport protocols, TCP and UDP. TCP tries to have control of message delivery, so if a router drops a packet, TCP will resend it until it gets confirmation from the other end that the message is received. UDP is like our royal mail service, you have no idea of what happened to the message. If you’re lucky, it will reach the destination.

Interactive realtime conversations are different from downloads

Protocols like HTTP and SMTP for web and email can use TCP and rely on retransmissions, since there is no realtime hurry. A few retransmissions won’t bother you much, but missing data in an e-mail message or web page might be very irritating. For media, especially interactive media, the situation is very different. Retransmissions will not help, since we propably already played the message in your speakers. We can’t reinsert part of a second worth of audio in the media stream after you’ve already enjoyed listening to it. For interactive media, like a phone call, we’re also in a hurry. We can’t wait for missing packets to arrive, since if we delay media too much, the call will break down. You will hear this and if it gets too hard, switch to James Bond walkie-talkie mode. “over”, “over, roger that”, “over and out”…Media for most of the standard VoIP protocols use RTP, the real time protocol. RTP is a way to send media over an IP network. Each message has a time stamp that tells the receiver where the payload fits into the media stream. If a packet is lost, the receiver will discover a glitch in the time stamps. Some receivers insert some noice to prevent you from discovering the issue. Of course, if there are too many packets lost you will notice. 20 milli-seconds here and there will propably go un-noticed in most cases.

Introducing RTCP – the bi-directional reporting system

RTP has a companion protocol called RTCP, the Real Time Control Protocol. This protocol is an out-of-band communication channel between the sender and the receiver. In a phone call, both ends are of course both receiving and sending.  RTCP is used to send reports to the other end, saying “I have sent xxx packets since we started and received yyy packets”. Both ends can then compare data and calculate the packet loss. The reports also include time stamps, so that the round trip time, the time for a packet to travel between the devices, can be measured. There are many pieces of data, and also a standard for extended reports that will deliver a bit more data.If a device, like an Asterisk server, collects this data, we can measure not only the quality of a particular phone call. We can gather data and get hints about issues with SIP trunks to a particular provider, about users on weak WiFi networks in hotels and a lot of other situations. We could potentially deliver this data in real time and alert users when the link is really bad, midcall.There are also advanced codecs that are adaptive to the situation and could use this data in real time. If the network shows signs of problems, the codec can try to change the flow of data – size of packets sent, rate of packets or other properties, like error correction and packet loss concealment.

Project Pinefrog – improvements to Asterisk’s RTCP support

Asterisk today has a very simple implementation of RTCP reports that isn’t very useful, but still a very good starting point. I’ve been working to make it a bit more useful by sending reports over manager, storing quality data in a database and also trying to improve the NAT support for RTCP. I’ve been testing a large number of phones to see how they have implemented RTCP and how Asterisk handles the received data. Hopefully, the turnout will be a large improvement and help us all in getting better and managed quality for our Internet telephony.This work is sponsored by a few companies in the Asterisk community who answered my earlier call for sponsorships.  I am always happy when things work out and Asterisk users step forward and contribute to the process of creating a better version of Asterisk. Being able to get quality data about the calls is a huge improvement for all of us that use the Internet as a transport for our telephone calls. As always your feedback is as welcome as the RTCP feedback on our SIP calls!

A named ACL is an Access Control List that can be manipulated after configuration and live in it’s own name space. The NACL module manage a list of NACL objects that can be used by other modules, like channel drivers, manager and dialplan apps.

Several SIP devices can share the same access control list and there will be one for the whole SIP channel. An external application that reads the security events in 1.8 can manipulate the NACLs in real time through AMI and block/unblock devices. There’s also an API so that Asterisk modules can modify NACLs internally. Applications can be added, so that NACLs can be manipulated through the dialplan. Call in, identify yourself and add yourself to an NACL for the next call…

Amongst the future ideas are NACLs that can be set by referring to a DNS name and use the DNSmgr to stay up to date with DNS. That requires some changes to the ACL.c api that will happen in the trunk version only.

I have also been playing with the idea of having a callback so that an app will know when a NACL is matched or some sort of counters to measure activity per time period and trigger alarms. Kamailio has one implementation of something like this in the pike module.

A lot of security-related ideas for Asterisk has been based on named ACLs, so I thought that was a starting point and a good holiday hack :-) The code is in the deluxpine branches for your testing!

Feedback and comments are, as always, welcome./olle

Yesterday, Russell Bryant finally made up his mind and confirmed on the asterisk-dev mailing list that the next release of Asterisk will be 1.8, which will also be a Long Term Support (LTS) release. This also means that the 1.4 is now officially classed as a LTS release too.

What is a long term release, LTS?

Long Term Releases are releases that are going to be supported for four years. Standard releases, like 1.6.x, are going to be supported for one year, with one additional year of security fixes. This means that the support for 1.6.0 will cease in october 2010. There’s a new release schedule on the Asterisk.org web site that explains the details.

Open Source Projects have to be easy to understand and use

I feel that this is a very good solution for the whole Asterisk community and that we all will benefit from it. I have personally not been happy with the 1.6.x release schedule, which has been very misunderstood and has confused a large group of users. Hopefully, we can now continue with a release schedule that the world understands and that makes sense for everyone. While I understand the need for releasing quicker than we’ve done in the past, the detail about the naming, the actual release numbers (1.6.0, 1.6.1 etc) was very hard to explain to people. With years of experience of doing Asterisk and VoIP training, I have a lot of respect of the need of being able to easily explain things, from configuration details to release schedule…

Time to focus on defining Asterisk 1.8

Now we, the Asterisk community, need to focus quickly on the new release and plan what’s going in there. If you have code for new features lying around (as I have tons of in various branches of my svn repository), now is the proper time to step forward, contribute it to the bug tracker and get it evaluated, discussed and maybe finally included in Asterisk. Whatever goes into 1.8 at release time, will be what we will have for production use for  a long time.

Please dedicate time for testing during Q1 and Q2 2010!

We also ask you to dedicate time during next year to help the Asterisk project with testing. You don’t have to be a developer to test – and we need tests of everything from documentation to configuration and technichal issues. We don’t have all of the equipment you have, we don’t have your dialplans, we don’t have all the applications you integrate Asterisk with. If Asterisk is important to your organization, please make sure that you dedicate time during the first half of 2010 to do regular testing of the new release betas and release candidates. We do need your help to make Asterisk 1.8 a good release, worthy to replace the 1.4 as a new LTS release.If you’re a member of a Linux or Asterisk group, please help in organizing Asterisk 1.8 test-partys. If you need help with ideas, please contact our community liason, John Todd. Meeting other Asterisk users, testing stuff together is one of the best ways to expand your knowledge of Asterisk. Sharing ideas and how-to’s in real time while setting up test labs and scenarious is really, really fun.

Asterisk 1.8 will make a difference

Asterisk has added a lot of new features and internal scalability and stability since 1.4. The 1.6.x releases are to me test releases to show and run practical tests with all of these changes. The core has changed, the API’s has changed and the internal PBX is practically new. We’ve proven scalability to over 10.000 calls on one server. We’ve proven interoperability with many, many products out there. We’ve changed the way we do development and we’ve moved Asterisk into the world of non-PSTN wideband audio. Of course, there’s a lot of more things we can do, but if we consider all of the changes since 1.4, Asterisk 1.8 LTS will be a really cool telephony toolkit.

At the first Astricon I was very happy to see Marc Blanchet as one of the attendees. I knew he was one of the IPv6 gurus and wanted someone to show some interest in Asterisk and IPv6.Well, he did not only get interested in it, but started coding on it. The results have been available for quite some time at http://www.asteriskv6.org/ and Marc has tested it at several SIPits for interoperability.

This patch is very large and affects large areas of Asterisk. In order to support IPv6, we need to update the way we interact with sockets, with DNS, with URI’s. The SIP channel needs to handle multiple UDP as well as TCP sockets in both protocols. The ACL’s we use for all VoIP protocols and manager needs support for IPv6. And much more.

Marc hasn’t been able to spend time to keep it up to date with the everchanging trunk. I feel we need to move this forward and try to divide the large patch into smaller pieces that can be reviewed separately by the developer team and  be merged gradually. First, Marcs branch needs a serious overhaul to get up to date with trunk.

In order to work on this, Marc and I needs funding .I have a few interested parties, but need more interested parties that can commit to funding during the first half of 2010 for this project.  It’s not a small task, the current estimate is at least one month’s work for each of us for updating, cutting it up, merging, going through the review process, testing and finalizing with new tests at SIPit or a similar event.

If your organization is interested, please let me know off list and we’ll discuss from there. My e-mail is as always oej@edvina.net. Please don’t hesitate to mail me with any questions you might have about this project.

/Olle

In a surprising move, Digium in partnership with Edvina today released a new channel driver for Asterisk, chan_tweet. The driver connects seamlessly to several microblogging platforms, including Twitter, Facebook, Laconi.ca/Identi.ca and GSM text/SMS. The main feature of this new module is to add audio and video capabilities to microblogging, making the popular microblogging networks a new platform for VoIP and IP realtime communication.

- “I have seen that the microblogging solutions building on the social network infrastructure have had enourmous unexploited capabilities”, says Mill Biller at Digium, “I’ve used it for a long time both personally and for the company and we realized early that by adding IAX2 support, we could now take these platforms one giant leap forward by adding realtime multimedia. I can now spend evenings chit-chatting in audio and HD-resolution video with all my audience around the world instead of sending short text messages. It’s truly awsome!”

Digium contracted Edvina in Sweden, a well-known company in the Asterisk community and long-term Digium business partner, to build this solution. Edvina has many years of experience in building large-scale IAX2 networks, as well as doing development on the IAX2 support in Asterisk.

- “IAX2 recently was published in an IETF RFC and we’re pushing it heavily in all VoIP forums.” says Olle Johansson of Edvina, “We’re hoping that the IAX2FORUM will get a lot of new members that are willing to adopt this technology for their intranets, microblogging services and VoIP infrastructures. In the coming month, we will present more information about new partners with more than 100K users that are going to switch from old technologies, like Hype, SIP and H.323. All of these protocols failed, either because they where proprietary or simply became too complex. SIP currently has more than 5.000 pages of documents describing all the features of the protocol and there’s no single implementation of all of these to test with. Considering the protocol being over 10 years old, this is a sad story.”- “We’ve done our best to fix the Asterisk SIP channel support for customers, but the customer base has been shrinking as more and more converted their networks to IAX2 and now, there’s simply no one interested in us doing that work. We’ve stated over and over again that the SIP channel in Asterisk is broken and no one can prove us right or wrong, because the protocol is just too complex.”

The Microblogmedia platform

The Microblogmedia(TM) platform, developed by Digium and Edvina, let’s users use any microblogging network to set up multimedia sessions. By compressing an IAX2 call setup event in the microblog message, web browsers and clients will connect automatically peer-2-peer if possible, or through the MicroBlogMediaRelay network that supports seamless NAT and firewall traversal by using automatic IPv6 tunnels.Asterisk 1.6.3, released later this month, will support this feature in the IAX2, H.323 and maybe in the old SIP channel (that is now marked deprecated). There is work on adding this feature to ISDN calls, by using messages in the D-channel for tunneling the IAX2 call setup messages. Digium’s VoxSwitch will support this feature in the next release, planned for q3 2009.

Ending the Hype project

In the same press release, Sock Stevens, product manager at Digium finally acknowledged that the Hype channel driver that was launched at Astricon 2008 will not be released after all.

– “We found only one partner to test interoperability with, and that’s not enough to make sure the channel driver being compatible with the protocol. And the protocol wasn’t published in any RFC at all, or any other document. So we finally gave up. We’re now dedicating resources for the new chan_tweet project and enhancing presence support in our IAX2 solution. With the installed base of IAX2 and the new MicroBlogMedia platform, this will be an even more impressive solution, reaching millions of IAX2 users in the enterprise as well as public sector and homes.”

Technichal factoids

  • chan_tweet is the result of the project labelled “Codename orangepeel” amongst the development team and builds on the new “Pinemango” architecture. This is the first channel driver not connecting directly to the Asterisk core, but to the Pinemango API over Adversion, the Ruby framework developed by Phil Jaysip.
  • The MicroBlogMediaRelay IAX2 platform is an open distributed network that builds on IPv6 and a facebook application, thus using the enormous bandwidth provided for free by the Facebook(TM) platform
  • chan_tweet will be released with the core module in Open Source, but with a license exception for plugin developers to add proprietary modules, like the Wireless Village plugin provided by the 3GPP project and the Unistim Microblog Solution by Nertol Networks.

For more information, please do not contact Digium sales.

To be released: 2009-04-01

« Previous PageNext Page »