• Home... >
  • Blog >
  • AWS Spot Blocks – New Way to Reserve Spot Instances in QDS



AWS Spot Blocks – New Way to Reserve Spot Instances in QDS


Co-authored by Goden Yao, Product Manager, Qubole and Somya Kumar, Member of the Technical Staff, Qubole

Qubole Data Service (QDS) now supports a new way to reserve AWS Spot instances. As of today, Qubole customers will be able to configure Master and Slave nodes to be Spot blocks.

Spot blocks are instance types that can be reserved for a predefined duration–between 1 to 6 hours. Since Qubole customers’ average cluster life cycle is under 6 hours, this gives them an option to guaranteed uninterrupted workload execution with relatively cheaper price, but higher stability compared to pure Spot nodes. Compared to Spot nodes, Spot blocks provide continuous computation time especially for applications that are not stateless and have no checkpoints generated in its execution.

For detailed documentation on using AWS Spot instances and Spot blocks in QDS, click here.

In QDS, you have the flexibility of choosing different cluster compositions for both Master and Auto-scaling nodes.


Original Cluster Compositions
  Master / Min Slave   Auto-scaling Slave
  #1   On-Demand   On-Demand
  #2   On-Demand   Spot
  #3   Spot   Spot
New Cluster Compositions Including Spot Blocks
  Master / Min Slave   Auto-scaling Slave
  #4   Spot Blocks   On-Demand
  #5   Spot Blocks   Spot
  #6   Spot Blocks   Spot Blocks


In the above:

  • If you know the cluster’s life cycle is within 6 hours and have an estimate about the duration of your workloads, you may choose #4 over #1 (same for #5 over #2) to leverage Spot blocks to achieve the same stability that On-Demand instances have as well as benefit from the lower cost of Spot blocks.
  • #6 is the only composition we would recommend to use Spot blocks for auto-scaling. The reason being when auto-scaling kicks in, we need to set a duration for Spot blocks request. If you choose Spot blocks for Master, Qubole will know the time remaining and use that time for autoscaling requests. For example, let’s say you choose #6 as your cluster configuration and duration for Master Spot blocks is set to 6 hours. In this case, after running for a couple of hours, let’s assume an autoscaling event occurs–when this happens, QDS will determine that the cluster life cycle only has 4 hours left (6-2=4). Therefore, when requesting Spot blocks for autoscaling, it will put in request for 4.


Spot blocks pricing is based on the requested duration and the available capacity; it’s typically cheaper than On-Demand nodes, but usually cost more than Spot nodes. Spot blocks and Spot instances are priced separately; view current spot pricing on AWS and select “Defined Duration for Linux” tab to learn more.

Note: Spot blocks work very similar to Spot nodes where users specify bidding parameters including max price. QDS uses 100% On-Demand price to bid for Spot blocks to make sure that total cost doesn’t exceed what you might pay for On-Demand nodes in the worst scenario.

In Summary

Spot Blocks provide a balance between cost and reliability for users who have the need to keep the cluster running for a definite period of time while keeping the costs at the minimum.

If you’re interested in QDS, sign up today for a free trial! To get an overview of QDS, click here to register for a live demo.