Netflix's utilization of EMR (Elastic Map Reduce) Hadoop on the cloud is an example that demonstrates Vogel's view on the engineering and business aspects of analyzing big data. With a subscriber base (50 million[1] as of July 2014) growing at unpredictable and accelerating rate, Netflix cannot manage data in their own data centers. The company stopped building their own data centers and shifted to a business model of using AWS (Amazon Web Service) as a source of storage and processing capacity[2].

In Werner Vogels' view, cloud services solve the engineering problem of dealing with big data[3]. Instead of wasting resources on IT infrastructure, businesses can focus on delivering differentiating business values. AWS allowed Netflix to focus on their value as a video streaming service provider.

Specifically, Netflix used thousands of EC2 (Elastic Compute Cloud) instances and petabytes of S3 (Simple Storage Service) in AWS to encode movies in batches, to process and analyze logs, to stream videos in a scalable manner, and to deploy a personalized movie choosing algorithm[4]. Hadoop EMR technology enabled Netflix to run the analytics and improve their personalized recommendation service.

Netflix had to re-architect their codebase and build their own tools to adapt to the cloud platform. These tools facilitate flow of data through every stage in their data pipeline. Encoded content, user details and logs are collected and stored in AWS. The data are analyzed in the cloud itself and they are made available for streaming in a variety of consumer electronic devices at a global scale. Thus Netflix was able to manage the business problem of making a growing collection of videos highly available and portable based on personal interests.

References

1. http://money.cnn.com/2014/07/21/technology/netflix-subscribers/
2. http://aws.amazon.com/solutions/case-studies/netflix/
3. https://www.youtube.com/watch?v=oNTp5spjv0w
4. http://www.zdnet.com/the-biggest-cloud-app-of-all-netflix-7000014298/