Cloud Control: Efficient Hadoop ETL Processing

November 9, 2016 Optimizing your use of AWS Spot Instances can save a ton on infrastructure costs, but how much work is it to constantly monitor fluctuating prices? And how to you deal with losing spot nodes in your cluster without it impacting your running jobs? At BloomReach, aggressive spot utilization saves up to 85% within their Big Data ETL environment. And no one at BloomReach is actively monitoring or bidding — it’s all automated. In this session, you’ll hear from Jorge Rodriguez, Tech Lead in BloomReach’s data platform team, to learn how they set up their ETL environment to take full advantage of Spot Instances, and how they leverage autoscaling with Qubole to get the most out of their Spot usage while simultaneously eliminating the drawbacks that make Spot Instance use so complex.

Previous Video
Qubole at Big Data World London, 2018
Qubole at Big Data World London, 2018

Big Data World 2018 in London was an action-packed couple of days that saw data enthusiasts from across the...

No More Videos