Verizon

Live Streaming Dashboard: Built, Delivered and Implemented in 60 Days

Snapshot

Verizon engaged Spark Digital to build a dashboard to enable a live-event streaming partnership with ESPN. Within 60 days, we were able to produce an MVP that supported event scheduling, monitoring and ad insertion and delivery.

The MVP we delivered performed beyond expectations, with scalability to support hundreds of simultaneous events with millions of concurrent viewers. After a successful implementation of the MVP, we continue working with the client to add features and additional functionality.

Business

Challenges

Verizon needed to build a platform to support streaming live events from ESPN to mobile devices. The application needed to allow operators to schedule and receive the live event from clients, monitor stream quality and, most importantly, trigger ad insertions during live events.

As a revenue center, and with the need to deliver ESPN content seamlessly, the system had to be bulletproof; availability, reliability, and responsiveness were mandatory while hiccups and distortions were unacceptable.Scale was also critical given that audiences would swell into the millions for many live events.

Results

Simply: the dashboard we built supports a new and growing revenue stream for Verizon and their broadcast partner. Together they’ve found a new way to make money and engage their customers/viewers with compelling content.

The dashboard also allowed the client operate more efficiently and profitably in these ways:

The scheduling feature puts the staff and QC resources where they need to be without redundancy while ensuring a flawless streaming experience for customers.
The scheduling feature also makes it possible to predict the load so bandwidth can be dialed up and down along with the demand.
The platform not only performs the ad delivery function, but it makes that process of uploading and scheduling and distributing ad inventory easy for operators.

Technology

Challenges

Time
When Verizon and ESPN teamed up, it was an opportunity to for them to develop a new revenue stream and deliver a valuable experience to subscribers. One little problem: the platform needed to be built and ready for action in 60 days. Clearly, we needed to ramp up a team of experienced engineers quickly and get them engaged with client-side systems engineers immediately.

Scale & Load
The platform needed to be capable of handling hundreds of live events at a time with viewership in the millions, it also needed to the ability to scale down to reduce costs. This required we implement dynamic scaling using cloud-based services.

Live & Real Time
There is little margin for error in live event streaming. Small interruptions or distortions will quickly limit the quality of the viewing experience. The dashboard we built needed to quickly identify problems and offer operators a quick solution. Further, the system needed to allow deployment of new features even while the system is running.

Availability
The most significant spikes in load occur over the weekend and after office hours. With the dashboard being critical to quality content delivery, Verizon needed to us have engineers on-hand monitoring the dashboard just in case something went wrong. Because our team is nearshore this required nights and weekends but not for anyone to work in the middle of the night.

Results

Streamed hundreds of events equalling more than 2.2M hours of live events, some having more than 150,000 concurrent viewers.
Integrated with Verizon’s live-feed services and APIs, wci required Spark Digital to implement flexible but highly secured authentication mechanisms.
Developed an Angular.JS-based front-end operating dashboard, allowing operators to preview multiple channels from different live feeds at the same time and modify event metadata on the fly. Within seconds of detecting an issue or live signal crash, our alert systems would failover to the secondary feeds.
Used Node.JS as part of the backend tech stack, including a combination of AWS RDS and MongoDB to achieve high responsiveness and performance.
Hosted on AWS, and leveraged additional AWS services such as Elastic IPs, Elastic Load Balancers and Simple Queue Service, among others.
Provided documentation and training sessions to support client-side implementation. We compiled and shared comprehensive documentation relating to system architecture and development, how to deploy updates to different environments, and how to diagnose issues. Finally, we ran a series of training sessions to ensure proper and effective knowledge transfer.

Our Approach

With a short time to complete the project, we worked quickly to bring together a team of experienced engineers with the skills to get the project done. Then, we worked with the client to quickly define the objective, and write epics and user stories while creating a long-term roadmap for project completion.

From the very beginning, it was important to define critical features, separate them from the ‘nice-to-haves’ and write a product roadmap that would result in a viable MVP within the 60-day deadline. We worked aggressively to meet those goals and support a successful product launch.

After the MVP launched and was successfully performing critical functions, we began a new phase of the project, working with the client to define and build additional features and add scalability through a continuous delivery model.

The Team

Within one week we assembled an experienced team of 8 engineers including existing employees and new hires from our nonstop talent acquisition engine. We assigned a Development Manager who takes full responsibility for project success and is the primary client contact. We also assigned two Tech Leads who worked together with the client on sprint planning and to make sure milestones were met. Finally, we had an additional five developers working full-time to meet the 60-day deadline.

Working with the Client

At the beginning of the project, we sent our tech leadership to meet face-to-face with the client at their corporate headquarters. In the early days we worked together to write epics and user stories that would drive the development approach. We also used that time build the relationships between engineering teams that would carry us through an intense development cycle while reducing client stress and worry.

The project was run through a typical agile process with daily scrums and weekly backlog sessions. During the product launch, we brought in a large support team to run QA and engineers to step in case of glitches (of which there were none). Ongoing we have a team on standby over the weekends when the load is most intense.

Case Studies