Big data-as-a-service provider Qubole Inc. has open-sourced its Quark cost-based SQL optimizer that simplifies and optimize access to data across multiple hosts.

Quark essentially chooses between popular big data open source query systems such as Hive, RedShift and Presto/Impala to select that which will deliver the best query results. “All of these query engines are good at some things and bad at others,” said Ashish Thusoo, co-founder and CEO of the 90-employee Qubole. “Subsuming that intelligence allows the machine to decide what engine is best used for that query.”

Quark addresses the problem of duplication caused when data sets are aggregated and copied to multiple analytics systems to balance performance and cost, making it difficult to efficiently query the data. Quark optimizes access to data by managing relationships between tables across an organization’s databases.

Quark models relationships between datasets using well-known database concepts like materialized views and OLAP cubes, enabling data analysts to automatically take advantage of the fastest and most efficient data set for their queries. Qubole provides platforms and tools for data-driven tasks that hide the complexity of SQL and back-end processing engines.

The Quark project is now available in open-source on GitHub, as well as in a SaaS implementation via the Qubole Data Service.

Thusoo and co-founder Joydeep Sen Sarma have authored or participated in several prominent open source projects, including the Apache Hive Project.  Prior to founding Qubole, Thusoo was head of the data infrastructure team at Facebook.