Blog

 
 
 

The Value of Auto-scaling

 

Intro

In a recent blog post, we benchmarked auto-scaling and demonstrated that an auto-scaling cluster was a lot less expensive and only a little bit slower than a static, max-sized cluster. In this post, we decided to quantify this benefit in terms of dollars and cents. Based on our results, we estimate that auto-scaling is worth over $300,000 a year for one of our customers for just one cluster.

Screen Shot 2016-08-16 at 9.45.04 AM

Methodology

Cost of Node Hours

First, recall that we based our benchmark on a real customer workload. We analyzed the cluster of a customer that runs a major consumer web application. This customer was running an auto-scaling cluster which scaled between about 100 to 400 nodes of the instance type c3.8xlarxge. According to Amazon EC2 pricing, c3.8xlarge instances cost $1.68/hour.

Our previous benchmark found that a static, max-sized cluster used on average 32% more nodes every hour than the cluster which auto-scaled. Our customer cluster had a max size of 400 nodes. By extrapolation, auto-scaling would reduce the actual average usage on the cluster to that of a 303 node cluster. This 97 node difference translates into $162.96 saved per hour. Multiplying that by the 11 hours that the cluster was active results in a savings of $1,792.56 per day in EC2 hours due to auto-scaling.

Cost of Analyst Time

However, we must also take into consideration the opportunity cost incurred by the analysts. In our benchmark, we found that auto-scaling was not quite as fast as the static, max-sized cluster. This makes sense because the static cluster always has its nodes ready to use, while auto-scaling only kicks in when the load on the cluster increases.

To estimate the cost of analyst time, we started with a ballpark hourly salary for a data scientist. According to Glassdoor, the average data scientist’s annual salary is about $114,000 a year. Taking into account the costs of things like employee benefits and allocated operating costs*, we estimate the average cost to a company of employing a data scientist for a year to be $171,000. Estimating that a data scientist works about 50 weeks a year and works 40 hours a week, this equates to 2,000 hours a year, resulting in an average cost of $85.50 per hour.

Our last analysis found that the static, max-sized node cluster ran 10% faster than the cluster which auto-scales. This means that for every hour a query ran on the max-sized cluster, the same query on the auto-scaling cluster would take six minutes longer to return. We found that for the cluster we analyzed, the total query run time was 68 hours**. This means that had the cluster been max-sized at 400 nodes instead of auto-scaling, the total query return time would have been 6.8 hours shorter***. Taking into account the hourly salary of an analyst, we calculated that the total extra cost of the auto-scaling in terms of analyst time was $581.40 per day. This number isn’t small, but it is far exceeded by the $1,792.56 saved in terms of node hours.

Value of Auto-scaling

Finally, we subtracted the $581.40 lost per day in analyst time from the $1,792.56 saved per day in node hours, which resulted in a total of $1,211.16 saved per day. Multiplying this number by the 250 work days in a year means that auto-scaling is worth $302,790 per year.

What about other clusters?

But what if you’re not running a 400-node cluster? What will your savings look like then? It turns out that even for smaller customers, the value of auto-scaling can be significant.

We took a look at two other clusters on the same day for two other customers. The first customer, a major US-based retailer, runs an auto-scaling cluster of up to 200 c3.8xlarge nodes, for whom auto-scaling is worth over $100,000 annually. The other customer is a major multi-national enterprise technology provider; for their auto-scaling cluster of up to 60 r3.4xlarge nodes, auto-scaling is worth almost $40,000 annually on just one cluster.

We’d love to show you the power and flexibility potential of auto-scaling. What will auto-scaling be worth for your organization and your use case? You can get started by checking out our 15-day free trial.

* Fully-loaded markup estimate is 50%.

** Note that this is not the same as the cluster run-time. This number is the result of adding together all of the return times for all of the queries in the cluster for that day.

*** It should be noted that a 6.8 hour difference in query return times does not literally translate into 6.8 hours of data scientist time wasted. When waiting for a query to return, data scientists can work on other projects. However, to be conservative, this analysis treats query run time as equivalent to wasted data scientist time. This actually decreases our estimate of the value of auto-scaling since we assume the decrease in query performance is felt directly in productivity.


Leave a Reply

Your email address will not be published. Required fields are marked *

Are you human? *

 
 
 
 
clear