Mobile Conferencing Meets Global Demand with New Infrastructure and Deep Automation on AWS

Executive Summary

Founded in 2012 and built from the ground-up for mobile users, Voxeet is disrupting conferencing and collaboration for mobile workers and distributed teams. The company's proprietary TrueVoice 3D surround sound technology and collaboration capabilities makes the “anywhere” conference room possible — giving distributed teams a competitive advantage for doing business. Voxeet's fast-growing user base spans 121 countries in industries including technology, advertising, financial services, legal, and healthcare. The company has won multiple awards including the DEMO God award at DEMO 2012. Led by seasoned business technology veterans with expertise in web communications, collaboration, social networking, and telecommunications, Voxeet is headquartered in the San Francisco Bay Area with offices across the globe, from Silicon Valley to Bordeaux, France. Voxeet is available on Web, iOS, and Android. For more information, visit Voxeet on the web.

The Challenge

In February 2016, Voxeet launched the fourth iteration of its mobile and Web-based audio conferencing system, designed to give companies an audio conferencing and collaboration system tailored to include mobile users. Version 4 of Voxeet includes several new collaboration features. Participants are now able to record conversations, and can also share files and photos. They will also be able to make specific channels, and categorize saved conversations, files, and chats. Voxeet has received multiple awards from industry publications for its unparalleled sound quality and innovative features. As a result, Voxeet is experiencing tremendous growth across the globe.

Voxeet was already taking advantage of the AWS Cloud with their application running in a single AWS Region. They were leveraging DynamoDB, ElastiCache, Simple Storage Service (S3), and Elastic Compute Cloud (EC2) services from AWS.

Looking at their accelerated growth patterns, Voxeet determined that infrastructure expansion and deep automation capabilities were going to be the key factors in meeting explosive global demands and protecting their reputation for delivering the highest quality services to customers around the world. Their infrastructure needed to be deployed into multiple regions to ensure performance and lower the communication latency. Deployments and releases needed to be fully automated and elastic to provide the highest levels of efficiency and scalability.

Global Presence:

ClearScale Solution

Voxeet partnered with ClearScale to design and deploy a highly scalable infrastructure across multiple geographies. Voxeet needed to provide the fastest service possible to their global customer base. The solution needed to be elastic for cost efficiencies and automated to reduce management overhead. Voxeet wanted to take full advantage of AWS and infrastructure-as-code practices to ensure predictable release and deployment processes.

ClearScale collaborated with Voxeet through an iterative requirements-gathering approach to ensure that all details were translated into the architecture design and supporting processes. Three AWS Regions (North America, Europe, and Asia) were selected to provide a global footprint to serve Voxeet’s customers. GeoIP based traffic distribution was implemented to route user requests to the closest AWS Region. AWS CloudFormation, Elastic Beanstalk, and Docker were chosen to provide infrastructure automation and application deployment capabilities.

Multi-Region Big Data Architecture

Voxeet’s production infrastructure is spread across three AWS Regions. A Virtual Private Cloud (VPC) is configured in each region that spans multiple Availability Zones (AZs). This configuration provides high availability and redundancy within each region. The light-weight session service is deployed across multiple AZs in the primary AWS region and is backed by DynamoDB and ElastiCache for performance and resiliency. Each of the three AWS Regions hosts heavy-lifting media servers across multiple AZs for high availability of services. These media servers are also configured as Auto Scaling groups to provide elasticity of the environment. Public facing ELBs distribute external user traffic to the Auto Scaling Session tier. Internal ELBs are deployed to distribute internal communications between Session and Media tiers. The Media tiers communicate with the Session tier and upload recordings to S3 through fault tolerant VPN connections.

Simplified Big Data Architecture Diagram:

Each Media Region required connectivity to the Primary Region, but are independent of other Media Regions. OpenVPN was deployed to provide secure cross-region connectivity. The primary region hosts a failover pair of OpenVPN instances operating in server mode. Media Regions hosted failover pairs of OpenVPN instances operating in client mode. This provides fault tolerant connections from Media Regions to the Primary Region.

Simplified OpenVPN configuration:

Cloud Automation

To establish a solid foundation for automation, AWS CloudFormation was used to enable infrastructure-as-code practices and Elastic Beanstalk implemented to manage deployments. These capabilities simplify operational processes and, through CloudWatch and CloudTrail, provide detailed logging of all changes performed in the environments.

CloudFormation enables the management and deployment of infrastructure components through code. Custom CloudFormation templates were developed to deploy basic AWS infrastructure components. These templates are reusable to deploy new Regions, or rebuild existing Regions. All infrastructure changes are made to the CloudFormation templates which are versioned and stored as code. This method eliminates manual configuration and provides ease of deployment and rollback.

Base components CloudFormation template deploys the following:

• VPC
• Internet Gateway/Routes
• Subnets/Multi-AZ
• Security Groups
• NAT/OpenVPN instances

Voxeet’s Session and Media applications are deployed from Docker images. Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries — anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.

Elastic Beanstalk has been implemented to coordinate deployments to multi-container Docker environments. Elastic Beanstalk automatically manages the deployment, from capacity provisioning, load balancing, auto-scaling, and application health monitoring. This reduces management overhead and streamlines operational activities. Another added bonus is that the Elastic Beanstalk service is free from AWS.

When new versions of Session or Media applications are released, Voxeet simply creates a new Elastic Beanstalk deployment and deletes the previous deployment. Since the Docker images are version controlled, rollback is as simple as deploying the previous code version.

Implementation and Migration Strategy

ClearScale practices Agile methodologies for development and implementation phases. This aligns well with the development practices used to deploy infrastructure-as-code and our iterative requirements gathering and architecture design processes.

The first step for implementation was writing code. ClearScale created a CloudFormation template to deploy the basic infrastructure components. The template was deployed in iterations until the infrastructure was configured per the design requirements. The next step was to set up Elastic Beanstalk. Again, we started with code and deployed iteratively until the applications were configured and functional testing passed. This was the process that created the Voxeet Staging environment.

We used the same CloudFormation templates and Elastic Beanstalk scripts to deploy Voxeet Production. As with all projects, additional configurations and functionality were required. Implementing these features follows the same procedure — deploy and test in staging followed by a release to production. Not only does this methodology increase the predictability of releases, it also increases the velocity at which code can be released.

Planning the service cutover started once staging and production environments were fully tested and all code committed. This was a detailed process carefully evaluating code freezes, database synchronization, and user data replication. After Voxeet and ClearScale agreed on a cutover plan, a “mock cutover” was executed to verify timing and steps in the process. Once the process was verified and tested, live service cutover was completed.

The Benefits

Voxeet has a highly available and infinitely scalable solution to meet the demands of their rapidly expanding customer base. In Europe and Asia, Voxeet realized communication latency reductions from 200ms to 80ms. This helps their proprietary TrueVoice technology to deliver crystal clear sound quality anywhere, anytime. Voxeet has the efficiencies of auto-scaling and automation to minimize the Total Cost of Ownership (TCO). IT management overhead has been reduced through leveraging managed AWS services. Also, by increasing the velocity of their deployment pipeline through automation, Voxeet is able to deploy iterations faster and more reliably. Voxeet can spend more time delivering innovative features, continue to wow their customers with best-in-class sound quality, and maintain a competitive edge over the competition.