Way back in August 2013, Gigaom wrote an article called “Is Qubole proving a demand for Hadoop in the cloud?” based on Qubole’s July 2013 numbers showing that we had used more than 100,000 nodes to process more than a petabyte of data, Gigaom concluded that that this “seems like a fair amount of activity for such a young platform”.
Now, fast forward 8 months, you can see amazing growth in our usage. In March 14’ alone we’ve used more than 200,000 nodes to process more than 20 petabytes of data. That’s right, up from over a petabyte in July to 20 petabytes in March as shown in the graph below. Now, that’s what you call a hockey stick!
Certainly, these kinds of numbers make an even more compelling case for a strong demand for Hadoop in the cloud. Our customers such as Komli Media reveal many reasons driving demand including huge reductions in monthly processing costs, scalability and faster job processing with our auto-scaling and S3 I/O optimization. We’re also seeing a lot of demand from customers who want to accelerate their time to value and to focus on business insights instead of the operational aspects of Hadoop.
Whatever the reason, we provide a reliable and scalable infrastructure to get the job done.
The only remaining question is whether we can continue our momentum. We believe that we are just seeing the tip of the iceberg as interest in running Qubole Data Service on Amazon Web Services is growing and our recent support for Google Compute Engine is drawing considerable interest. Stay tuned for our next update!