Ensighten: Building a world-class digital advertising analytics platform using Qubole

Leveraging Data for Targeted Advertising

Ensighten’s flagship product, Ensighten Marketing Data Platform, powers a number of mission-critical use cases spanning omni-channel personalization, mobile experience optimization, and attribution using technology spanning four categories:

By helping them make sense of the huge highway of tags and customer data flowing across all of their corporate web pages, off-site ads, and data vendors’ environments, Ensighten allows marketing departments to operate much more efficiently. In addition, they deliver safeguards to help with data privacy and security, more than 1100 turnkey vendor tag integrations, and direct ownership of digital customer data.

About Ensighten

Ensighten is the leader in enterprise tag management and marketing data solutions. Using a combination of technology, vision, and experienced leadership, they help enable Fortune 50 – Fortune 500 companies to securely manage and unify disparate marketing technologies and data sources in order to create meaningful customer interactions across touchpoints.

The Challenge

In their early stages, Ensighten relied heavily on Hadoop to generate reports for customers. As Ensighten continued to grow, they found that Hadoop was acting more like an application, and less like the platform that they needed. This meant that their ability to serve an individual customer was almost always reliant on a substantial amount of custom development, which drastically increased turnaround time and pulled developers away from improving Ensighten’s core offerings. As such, getting reports and data interactions to customers was taking way too long — weeks at best, and usually months, and being done at the expense of other important business activities. They were already using Amazon Web Services (AWS), and needed a solution that allowed them to automate Big Data analytics so they could turn reports around for customers faster.

The paradigm of storage separate from compute infrastructure is first-class within Qubole.

Ben Roubicek, Software Architect, Ensighten

Why Amazon Web Services + Qubole

Since Ensighten was already using AWS, they decided to rebuild their platform using Amazon Simple Storage Service (Amazon S3) and Apache Kafka (an open source messaging system designed for building real-time applications using streaming data) on Amazon Elastic Compute Cloud (Amazon EC2).


At the core of this application was the concept of schema management, which allowed them to think about their data warehouse as a catalog and keep the data warehouse developer- friendly, while also utilizing a standard rows and columns structure. Using their internal schema management tool, they are able to replicate that schema to Qubole, which references the data in Amazon S3 and publishes a table that can be queried in minutes, without any work from a developer.

They chose Qubole for two primary reasons. One, it allowed them to decouple their compute from their storage, which made them more flexible. “We can have all of our storage in (Amazon) S3, but have all our compute power very elastic,” said Ben Roubicek, software architect at Ensighten. Secondly, they found that it allowed them to handle user-level management and permissions in a role-based manner across a variety of Spark, Hadoop and Presto, and a variety of other open-source technologies. Before adopting Qubole, they found that increasing the proper authentication and access mechanisms for all of these services across their user-base was extremely time-consuming.

The Benefits

Ensighten has had several big wins since adopting Qubole as part of their AWS environment. On more than one occasion, a customer has been struggling to generate value out of the raw data that Ensighten provides them. In these cases, Ensighten leverages Qubole to help find insights for these customers and build them a custom reporting solution that meets their needs.

Another area where Qubole has been tremendously helpful is in the context of troubleshooting and diagnostics across their entire log file infrastructure. Because they capture 1-2 TB of log data per day, it was nearly impossible to pinpoint the cause of an issue cost-effectively in the past. With Qubole, they can analyze this log data much more quickly and inexpensively when compared to other solutions, which has been huge for Ensighten.

Perhaps the biggest benefit of adding Qubole to their AWS deployment was that it allowed them to place more emphasis on their core competencies. Because they have a true multi-tenant system that allows them to meet the needs of each of their customers without much custom development, they can focus on improving Ensighten’s core product offerings.

Download the PDF version of this case study.