It is common knowledge that data lakes offer the right architecture to support multiple use cases and tools, but can be operationally complex to implement… The post Data Discovery Tools – Qubole...

Now that you have read about how different components of Airflow work and how to run Apache Airflow locally, it’s time to start writing our… The post Apache Airflow Tutorial – DAGs, Tasks,...

In the past 2-3 years, Presto has set the bar for fast analytical processing in modern cloud data lake architectures. Qubole has offered a Presto… The post Presto on Qubole is 2.6x faster than...

Image credits: https://science.howstuffworks.com/terraforming.htm The Qubole Open Data Lake Platform Qubole is the open data lake company that provides a simple and secure data lake platform… The...

Running Large distributed Apache Spark clusters in the public cloud, that handle exponential increase in volumes of data to fuel analytics and machine learning (ML)… The post Logan: A Data-Driven...

Introduction Ad-hoc analytics and data exploration require compute resources that can process incoming jobs instantaneously and keep the response time low. Apache Spark is a… The post Cost and...

Columnar data formats have become the standard in data lake storage for fast analytics workloads as opposed to row formats. Columnar formats significantly reduce the… The post Columnar Format in...

Qubole is excited to announce the general availability of Managed Spot Block instances that provides up to 40% cost savings over On-Demand Ec2 Instances. Managed… The post Introducing Managed Spot...

As a general-purpose programming language, Python is universal. It’s quick and easy, but yet powerful with plenty of capabilities. It gives you an opportunity to… The post Boosting Parallelism for...

This is Part 2 of a 2 blog series on this topic.  You can read Part 1 here. Rollout Strategy:  We have different tiers in… The post Rails: Why Upgrading Matters – Part 2 appeared first on Qubole.

What’s keeping streaming data processing investments from yielding “speedy” results? There are multiple streaming data processing solutions out there but none are well equipped to… The post Lower...

Ruby on Rails (or Rails) is a web development  framework that gives Rails developers an optimized experience to write their (Ruby) code. Rails is one… The post Ruby on Rails: Why Upgrading Matters...

In an earlier blog post, we discussed the availability of Jupyter-based Notebooks for machine learning (ML) and analytics with a host of features that make… The post New Enhancements for Qubole...

This final part of the three part spark optimization series explains how a Spark application can be optimized for performance by using Qubole Sparklens. The… The post How to Optimize Spark...

With 90% of all the data in the world created in just the last 2 years, data scientists who historically worked on Python or R… The post Spark Packages – How to Manage Them appeared first on Qubole.

How to Optimize Spark Clusters on Qubole for Cost Reliability and Performance This second blog from the three part series explains how a Spark cluster… The post Spark Cluster Optimization for...

How Qubole Maximizes Spot Utilization and Reduces Costs One of our customers—a large enterprise cloud content management company—runs several sophisticated machine learning (ML) predictive...

Now that we know about Airflow’s different components and how they interact, let’s start with setting up Airflow on our workstation so that we can… The post How to Install Apache Airflow to Run...