Mike Olson, Chief Strategy Officer and Co-Founder at Cloudera, explains Apache Spark’s origins, its rise in popularity in the open source community, and how Spark is primed to replace MapReduce as the general processing engine in Hadoop.

Programming thousands of machines is no easy task.

One approach pioneered by Google is known as MapReduce.

MapReduce provides a programming model that simplifies programming thousands of machines by breaking down distributed programs into two steps: map, and reduce.